Method of processing fingerprint information

ABSTRACT

A method for processing fingerprint information includes dividing an input image that corresponds to at least a portion of a user fingerprint into a plurality of first regions; dividing a registered image that had previously been stored into a plurality of second regions; selecting a first matching region from among the plurality of first regions, and selecting a second matching region from among the plurality of second regions, by comparing the plurality of first regions with the plurality of second regions; and matching the registered image with the input image by comparing the first matching region with the second matching region.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims priority from Korean Patent Application No.10-2017-0010058 filed on Jan. 20, 2017 in the Korean IntellectualProperty Office, the disclosure of which is incorporated herein byreference in its entirety.

BACKGROUND 1. Field

Example embodiments relate to a method of processing fingerprintinformation.

2. Description Of Related Art

As utilization of various types of information technology (IT) devices,including mobile devices, is gradually expanding, various technologiesfor enhancing the security of IT devices have been proposed. In additionto existing password and pattern inputs, security technology based onusing the personal biometric information of individuals has been appliedto various IT devices. Among security technologies that use biometricinformation, fingerprint sensing technology has various positiveproperties in terms of manageability, security, economy, and the like.Thus, the application thereof has gradually increased.

SUMMARY

One or more example embodiments is to provide a method for processingfingerprint information, in which fingerprint recognition performancemay be improved.

According to an aspect of an example embodiment, there is provided amethod of authenticating a user based on fingerprint recognition,including: receiving, by a fingerprint sensor, a fingerprint of theuser; dividing, by a processor, an input image that corresponds to atleast a portion of the fingerprint of the user into a plurality of firstregions; dividing, by the processor, a registered image that haspreviously been stored into a plurality of second regions; selecting, bythe processor, a first matching region from among the plurality of firstregions and selecting a second matching region from among the pluralityof second regions, based on a comparison between the plurality of firstregions and the plurality of second regions; matching, by the processor,the registered image with the input image by comparing the firstmatching region with the second matching region; and authenticating theuser based on a result of the matching.

The method may further include determining, by the processor, aplurality of matching probabilities that correspond to similaritiesbetween each of the plurality of first regions and each of the pluralityof second regions; and determining, by the processor, the first matchingregion and the second matching region based on a highest matchingprobability from among the plurality of matching probabilities.

The determining the plurality of matching probabilities may includedetermining the plurality of matching probabilities by determining phasecorrelations between each of the plurality of first regions and each ofthe plurality of second regions.

A peak value of each of the phase correlations may be determined as theplurality of matching probabilities.

A peak value of each of the phase correlations may be detected togenerate correction values for performing a rotation transform of theregistered image.

The matching the registered image with the input image may includerotating the registered image, based on a first correction valuegenerated by comparing the first matching region with the secondmatching region; and matching the registered image with the input imageby moving the registered image that has been rotated based on a secondcorrection value, wherein the second correction value is generated bycomparing the input image with the registered image that has beenrotated.

The second correction value may be generated by comparing the firstmatching region with the second matching region after the registeredimage has been rotated based on the first correction value.

The first correction value may be generated by determining a rotationangle for the second matching region with respect to the first matchingregion by comparing a first frequency component of the first matchingregion with a second frequency component of the second matching region.

The method may further include determining, by the processor, a matchingscore that indicates a similarity between the registered image and theinput image, in an overlap region formed by the matching the registeredimage with the input image; determining, by the processor, an area ofthe overlap region; and determining, by the processor, whether thefingerprint is authenticated, based on at least one from among thematching score and the area of the overlap region.

The determining the matching score may include determining a normalizedcross-correlation between the registered image and the input image inthe overlap region.

The determining the normalized cross-correlation may include dividingthe overlap region into a plurality of sub-regions; searching minutiaein the overlap region; and determining the normalized cross-correlationby assigning a predetermined weight to each of a first sub-region thatincludes the minutiae and a second sub-region adjacent to the firstsub-region.

The registered image may include a plurality of registered componentimages, and the dividing the registered image may include dividing eachof the plurality of registered component images into a plurality ofregions.

The method may further include determining, by the processor, aplurality of matching probabilities of the plurality of registeredcomponent images with respect to the input image by comparing theplurality of first regions with the plurality of second regions; andselecting, by the processor, the first matching region and the secondmatching region by comparing each of the plurality of registeredcomponent images with the input image in a sequential order for whichthe plurality of matching probabilities are decreasing.

The method may further include comparing, by the processor, theplurality of matching probabilities of the plurality of registeredcomponent images with a predetermined threshold value; selecting, by theprocessor, registered component images having a matching probabilitythat is greater than the predetermined threshold value as candidateregistered images, and determining, by the processor, whether thefingerprint is authenticated by matching each of the candidateregistered images with the input image in the sequential order for whichthe plurality of matching probabilities are decreasing.

According to an aspect of another example embodiment, there is provideda method of authenticating a user based on fingerprint recognition,including: receiving, by a fingerprint sensor, a fingerprint of theuser; dividing, by a processor, each of an input image that correspondsto at least a portion of the fingerprint of the user and a registeredimage that has previously been stored into a plurality of sub-regions;finding, by the processor, from among the plurality of sub-regions ofeach of the registered image and the input image, a first sub-regionthat is included in an overlap region in which the registered imageoverlaps the input image and a second sub-region that is included in theoverlap region and is adjacent to the first sub-region, each of thefirst sub-region and the second sub-region including minutiae; anddetermining, by the processor, whether the user is authenticated, byassigning a first predetermined weight to the first sub-region and asecond predetermined weight to the second sub-region.

The first predetermined weight assigned to the first sub-region may begreater than the second predetermined weight assigned to the secondsub-region.

The determining whether the fingerprint is authenticated may includedetermining a matching score that indicates a similarity between theregistered image and the input image in the overlap region by assigningthe first predetermined weight to the first sub-region and the secondpredetermined weight to the second sub-region; determining an area ofthe overlap region, and determining whether the fingerprint isauthenticated based on at least one from among the matching score andthe area of the overlap region.

The matching score may be determined by assigning each of the firstpredetermined weight and the second predetermined weight when anormalized cross-correlation between the registered image and the inputimage is computed.

The registered image may include information that relates to minutiaeincluded in an original fingerprint image that has previously beeninput.

The method may further include dividing, by the processor, the inputimage into a plurality of first regions and dividing the registeredimage into a plurality of second regions; and matching, by theprocessor, the input image with the registered image to form the overlapregion, by comparing the plurality of first regions with the pluralityof second regions.

The matching may include determining a plurality of matchingprobabilities that correspond to similarities between each of theplurality of first regions and each of the plurality of second regions;selecting, based on a highest matching probability from among theplurality of matching probabilities, a first matching region from amongthe plurality of first regions and a second matching region from amongthe plurality of second regions; generating a first correction value tobe used for performing a rotation transform of the registered image bydetecting a peak value of a phase correlation between the first matchingregion and the second matching region; rotating the registered image,based on the first correction value; generating a second correctionvalue to be used for performing a translation transform of theregistered image after the registered image has been rotated, bycomparing the registered image that has been rotated with the inputimage; moving the registered image that has been rotated, based on thesecond correction value; and matching the input image with theregistered image that has been moved.

Each of the plurality of first regions and each of the plurality ofsecond regions may have an area that is greater than respective areas ofthe first sub-region and the second sub-region.

According to an aspect of another example embodiment, there is provideda method of authenticating a user based on fingerprint recognition,including: receiving, by a fingerprint sensor, a fingerprint of theuser; receiving, by a processor, a request for adding a partialfingerprint image that corresponds to at least a portion of thefingerprint of the user; determining, by the processor, whether to matchthe partial fingerprint image with a template that has previously beenstored, by comparing the partial fingerprint image with the template;and updating, by the processor, the template by adding the partialfingerprint image to the template, when the partial fingerprint image ismatched with the template.

The template may include a plurality of component templates, and each ofthe plurality of component templates may include at least one registeredimage that corresponds to a fingerprint.

The method may further include determining, by the processor, whether afirst component template that includes a first registered image matchedwith the partial fingerprint image is present by comparing the partialfingerprint image with the at least one registered image included ineach of the plurality of component templates; adding, by the processor,the partial fingerprint image to the first component template as asecond registered image when the first component template is determinedas being present; and determining, by the processor, whether the firstcomponent template is able to be merged with at least one additionalcomponent template by comparing the first component template withremaining component templates from among the plurality of componenttemplates.

The method may further include merging, by the processor, the firstcomponent template with the at least one additional component templatewhen the first component template is determined as being able to bemerged with the at least one additional component template.

The method may further include arranging, by the processor, theplurality of component templates based on at least one from among anumber of authentication successes, a number of registered imagesincluded in each of the plurality of component templates, and an area ofan overlap region formed by overlapping respective registered imagesincluded in each of the plurality of component templates with eachother.

When the partial fingerprint image is not matched with the template, anew template may be generated, and the partial fingerprint image may bestored as a registered image in association with the new template.

According to an aspect of another example embodiment, there is provideda method of authenticating a user based on fingerprint recognition,including: receiving, by a fingerprint sensor, a fingerprint of theuser; receiving, by a processor, an input image that corresponds to atleast a portion of the fingerprint of the user; determining, by theprocessor, from among a plurality of templates, whether anauthentication template to be matched with the input image is present bycomparing each respective one from among the plurality of templates withthe input image; and when the authentication template is determined asbeing present, facilitating, by the processor, an authentication of thefingerprint, and updating, by the processor, the authentication templateby using the input image.

When the authentication template is determined as being present, thefingerprint may be authenticated, and a number of authenticationsuccesses of the authentication template may be updated.

When the authentication template is determined as not being present, theauthentication of the fingerprint may be rejected.

The method may further include: after the authentication template isupdated by using the input image, determining, by the processor, whetherat least one additional template, from among the plurality of templates,matches with the authentication template that has been updated, and whenthe at least one additional template is determined as being present,merging, by the processor, the authentication template that has beenupdated with the at least one additional template.

A first number of authentication successes of a new template generatedby merging the authentication template that has been updated with the atleast one additional template may be set to be equal to a sum of asecond number of authentication successes of the authentication templatethat has been updated and a third number of authentication successes ofthe at least one additional template.

The method may further include arranging, by the processor, theplurality of templates based on a respective number of authenticationsuccesses of each of the plurality of templates.

The updating may include updating the authentication template based onat least one from among an area of an overlap region formed byoverlapping a registered image included in the authentication templatewith the input image, and a normalized cross-correlation of theregistered image and the input image calculated in the overlap region.

The authentication template may include a plurality of registered imagesand the overlap region may correspond to a region in which all of theplurality of registered images and the input image overlap each other.

According to another example embodiment, there is provided a fingerprintprocessing apparatus. The fingerprint processing apparatus includes afingerprint sensor configured to receive at least a portion of afingerprint as an input; and a processor. The processor is configured todivide an input image that corresponds to at least the portion of thefingerprint into a plurality of first regions; divide a registered imagethat has previously been stored into a plurality of second regions;select a first matching region from among the plurality of first regionsand select a second matching region from among the plurality of secondregions, based on a comparison between the plurality of first regionsand the plurality of second regions; match the registered image with theinput image by comparing the first matching region with the secondmatching region; and authenticate a user based on a result of the match.

The processor may be further configured to determine a plurality ofmatching probabilities that correspond to similarities between each ofthe plurality of first regions and each of the plurality of secondregions; and determine the first matching region and the second matchingregion based on a highest matching probability from among the pluralityof matching probabilities.

The processor may be further configured to determine the plurality ofmatching probabilities by determining phase correlations between each ofthe plurality of first regions and each of the plurality of secondregions.

The processor may be further configured to determine a peak value ofeach of the phase correlations as the plurality of matchingprobabilities.

The processor may be further configured to detect a peak value of thephase correlations in order to generate a correction value to be usedfor performing a rotation transform of the registered image.

BRIEF DESCRIPTION OF DRAWINGS

The above and/or other aspects will be more clearly understood from thefollowing detailed description taken in conjunction with theaccompanying drawings, in which:

FIG. 1 is a view illustrating an electronic device, according to anexample embodiment;

FIG. 2 is a diagram illustrating a fingerprint that may be recognized ina method for processing fingerprint information, according to an exampleembodiment;

FIGS. 3 and 4 are schematic drawings illustrating a method forprocessing fingerprint information, according to an example embodiment;

FIG. 5 is a flow chart illustrating a method for processing fingerprintinformation, according to an example embodiment;

FIG. 6 is a schematic diagram illustrating an image matching method,according to an example embodiment;

FIG. 7 is a diagram illustrating log-polar conversion, according to anexample embodiment;

FIG. 8 is a view illustrating a method for processing fingerprintinformation, according to an example embodiment;

FIG. 9 is a flow chart illustrating a method for processing fingerprintinformation, according to an example embodiment;

FIGS. 10 and 11 are views illustrating a method for processingfingerprint information, according to an example embodiment;

FIG. 12 is a diagram illustrating a user authentication method,according to an example embodiment;

FIGS. 13, 14A, 14B, 14C, and 14D are views illustrating a method forprocessing fingerprint information, according to an example embodiment;

FIGS. 15 and 16 are views illustrating a user authentication method,according to an example embodiment;

FIGS. 17 and 18 are views illustrating a method for processingfingerprint information, according to an example embodiment;

FIG. 19 is a diagram illustrating a registered image processing method,according to an example embodiment;

FIGS. 20 and 21 are flow charts illustrating a registered imageprocessing method, according to an example embodiment;

FIGS. 22A, 22B, and 22C are diagrams illustrating a registered imageprocessing method, according to an example embodiment; and

FIG. 23 is a block diagram of an electronic device, according to anexample embodiment.

DETAILED DESCRIPTION

Hereinafter, example embodiments will be described with reference to theaccompanying drawings.

FIG. 1 is a view illustrating an electronic device, according to anexample embodiment.

Referring to FIG. 1, an electronic device 1 according to an exampleembodiment may be a mobile device, and may include any of a smartphone,a tablet personal computer (PC), or the like, in an example embodiment.The electronic device 1 may include a display 2, a housing 3, a cameraunit (also referred to herein as a “camera”) 4, a key input unit (alsoreferred to herein as a “key input component”) 5, and a fingerprintsensor 6. In an example embodiment as illustrated in FIG. 1, thefingerprint sensor 6 may be integrated with a physical button that islocated in a lower end of the display 2, and may also be provided in aposition that is different from that illustrated in FIG. 1 or providedintegrally with the display 2, in other example embodiments.

As applications providing financial and payment services in electronicdevices such as smartphones, tablet PCs and the like have becomewidespread and the applications which are executable in electronicdevices provide purchasing functions for specific goods or services,demand for enhanced security performance has increased. The fingerprintsensor 6 may be easily implemented to have a relatively small size in alimited form factor, and since the probability that respectiveindividuals have the same fingerprint is extremely low, the fingerprintsensor 6 has been widely applied to various electronic devices.

However, if fingerprint recognition performance of the fingerprintsensor 6 is poor, a user who does not have user authorization for theelectronic device 1 may acquire access rights to the electronic device 1due to an erroneous sensing of the fingerprint sensor 6. Thus, invarious example embodiments, a method for processing fingerprintinformation, in which the convenience of legitimate users may beimproved via the fingerprint sensor 6 having improved fingerprintrecognition performance in order to prevent a user that does not haveuser authorization for the device from accessing the electronic device1, is provided.

The fingerprint sensor 6 may include a sensing unit (also referred toherein as a “sensor component”) which is configured to acquire an imageof a user fingerprint in various manners, and an image processing unit(also referred to herein as an “image processor”) which is configured toprocess the image acquired by the sensing unit. The method forprocessing fingerprint information proposed by various exampleembodiments may be executed in the image processing unit. The imageprocessing unit may be implemented as a separate processor connected tothe fingerprint sensor 6 or an application processor configured tocontrol operations of the electronic device 1.

FIG. 2 is a diagram illustrating a fingerprint that may be recognized ina method for processing fingerprint information, according to an exampleembodiment.

Referring to FIG. 2, a fingerprint 10 may be recognized by a fingerprintsensor, and in an example embodiment, the fingerprint sensor may have asmaller area than that of the fingerprint 10. In detail, in the case ofa mobile device, it may be difficult to provide a fingerprint sensorthat has a sensing area in which the entirety of the fingerprint 10 maybe recognized within a limited area. Thus, the sensing area of thefingerprint sensor may be smaller than the fingerprint 10.

For example, when a sensing region of the fingerprint sensor is smallerthan the fingerprint 10, fingerprint images 11, 12, 13, 14, 15, 16, and17 generated by the fingerprint sensor may cover a portion of thefingerprint 10. In an enrollment stage of obtaining the fingerprintimages 11, 12, 13, 14, 15, 16, and 17 from a user and storing thefingerprint images 11, 12, 13, 14, 15, 16, and 17 as a registered image,a plurality of the fingerprint images 11, 12, 13, 14, 15, 16, and 17 maybe input and stored. At this time, regions of the fingerprint 10 coveredby the plurality of fingerprint images 11, 12, 13, 14, 15, 16, and 17,respectively, may overlap each other. The registered image may bemanaged as a template, and a single template may include at least one ofthe fingerprint images 11, 12, 13, 14, 15, 16, and 17. One template mayinclude two or more of the fingerprint images 11, 12, 13, 14, 15, 16,and 17, and in an example embodiment, the fourth, fifth, and sixthfingerprint images 14, 15, and 16 having an overlapping region may beincluded in one template to be managed as the template.

In a verification stage of recognizing a user who has input thefingerprint 10, the fingerprint images 11, 12, 13, 14, 15, 16, and 17may be input and compared with a previously stored registered image or atemplate. At this time, a legitimate user may be erroneously recognizedas an unauthorized user, or an unauthorized user may be recognized as alegitimate user, depending on a position and angle of the fingerprint 10input by the user. Thus, in order to improve fingerprint recognitionperformance, various methods in which an input image may be matched witha registered image are provided.

FIGS. 3 and 4 are schematic drawings illustrating a method forprocessing fingerprint information, according to an example embodiment.In example embodiments to be described with reference to FIGS. 3 and 4,the verification stage, in which an input image is received by afingerprint sensor and is then compared with previously-storedregistered images, to thus recognize a user, may be performed.

With reference to FIG. 3, a user may cause a user fingerprint to makecontact with a fingerprint sensor 21 of an electronic device 20. Thefingerprint sensor 21 may have a predetermined sensing region, and maygenerate an input image 30 from a portion of the fingerprint that is incontact with the sensing region. In an example embodiment, the inputimage 30 may only cover a portion of the fingerprint, but not theentirety of the fingerprint. The electronic device 20 may compare theinput image 30 with a plurality of registered images 41, 42, and 43stored in a memory 40 of the electronic device 20, and thus, maydetermine whether or not the fingerprint that has contacted thefingerprint sensor 21 is authenticated.

The input image 30 generated by the fingerprint sensor 21 may bedetermined based on a direction and a region in which the user touchesthe fingerprint sensor 21 with a user fingerprint. The electronic device20 may compare the input image 30 with the registered images 41, 42, and43, respectively, to select at least a portion of the registered images41, 42, and 43 that have a relatively high correlation with the inputimage 30. An image processing method such as Fourier transform,log-polar transformation, or the like may be used to determine thecorrelation between the input image 30 and the registered images 41, 42,and 43.

With reference to FIG. 4, a registered image 60 having a relatively highcorrelation with an input image 50 may be selected, and the registeredimage 60 may be rotated by a predetermined angle according to a resultof correlation analysis, to thus generate a rotated registered image 70.Whether or not the input image 50 and the rotated registered image 70overlap each other may be determined, and an area of an overlap regionin which the input image 50 and the rotated registered image 70 overlapeach other, a matching score calculated for the overlap region, and thelike, may be calculated, to determine whether or not the fingerprintthat has generated the input image 50 is authenticated. In an exampleembodiment, the matching score may be a normalized cross-correlation(NCC) score.

In the example embodiment illustrated in FIG. 4, an entire region of theinput image 50 and an entire region of the registered image 60 may becompared in order to calculate a correction value which includes anangle required for rotation of the registered image 60. The correctionvalue may be obtained by applying image processing processes such asFourier transform, log-polar transformation, and the like, to the entireregion of each of the input image 50 and the registered image 60, andthen analyzing a phase correlation therebetween. In this case, regionsother than the overlap region that have a relatively high correlationwith respect to each other may act as a noise component in the phasecorrelation analysis of the input image 50 and the registered image 60.Thus, the accuracy of the phase correlation analysis may be degraded,which may lead to degradation of fingerprint recognition performancewith respect to the fingerprint that has generated the input image 50.

In various example embodiments, each of the input image 50 and theregistered image 60 may be divided into a plurality of regions, and eachof the plurality of regions may be subjected to image processing andthen phase correlation analysis may be performed. Thus, in the phasecorrelation analysis, the influence of the remaining regions excludingthe overlap region as noise components may be significantly reduced, andthe fingerprint recognition performance may be improved.

FIG. 5 is a flow chart illustrating a method for processing fingerprintinformation, according to an example embodiment. In an exemplaryembodiment, the method for processing fingerprint information to bedescribed with reference to FIG. 5 may be executed by a processor of anelectronic device that includes a fingerprint sensor.

Referring to FIG. 5, a method for processing fingerprint informationaccording to an example embodiment may begin with receiving an inputimage that corresponds to at least a portion of a user fingerprint by aprocessor, in operation S1001. The operation S1001 may includegenerating an input image from at least a portion of the userfingerprint that is in contact with the sensing region of thefingerprint sensor. In an example embodiment, the input image mayinclude a portion of the entire region of the user fingerprint.

The processor may divide the input image into a plurality of firstregions, and may divide a registered image that has previously beenstored in the electronic device, into a plurality of second regions inoperation S1002. For example, when the registered image is provided as aplurality of registered component images, the processor may divide eachof the plurality of registered component images into a plurality ofsecond regions. Each of the plurality of first regions may have the samearea, and each of the plurality of second regions may also have the samearea. In an example embodiment, the first region and the second regionmay have the same area.

In the process of dividing the input image into the first regions, atleast portions of the first regions may overlap each other. Similarlythereto, in the process of dividing the registered image into the secondregions, at least portions of the second regions may overlap each other.

When the region division is completed, the processor may compare thefirst regions and the second regions with each other in operation S1003.In operation of S1003, each of the first regions may be compared withall of the second regions. In an example embodiment, for example, whenan input image is divided into four first regions and each of tworegistered images is divided into three second regions, one first regionmay be compared with six second regions. For example, the imagecomparison process with respect to images of the first regions and thesecond regions may be performed a total of 24 times.

After the first and second regions are compared with each other, theprocessor may select a first matching region from among the firstregions, and a second matching region from among the second regions inoperation S1004. The first matching region and the second matchingregion may be the first region and the second region determined to havea highest correlation with each other in the comparison process ofoperation S1003.

The processor may compare the first matching region with the secondmatching region in order to match the input image with the registeredimage in operation S1005. For image matching in operation S1005, theprocessor may apply a rotation transform and a translation transform toat least one of the registered image and the input image. In an exampleembodiment, the processor may rotate and move the registered imagehaving the second matching region to overlap at least portions of theregistered image and the input image with respect to each other, andthus, may match the input image with the registered image.

The processor may determine whether the user fingerprint that hasgenerated the input image is authenticated, by using the matched inputimage and registered image in operation S1006. By matching theregistered image and the input image in operation S1005, the registeredimage and the input image may have an overlap region formedtherebetween. The processor may determine whether to authenticate theuser fingerprint that has generated the input image, based on at leastone of an area of the overlap region and a matching score calculated inthe overlap region.

FIG. 6 is a schematic diagram illustrating an image matching method,according to an example embodiment. In an example embodiment, the imagematching method to be described with reference to FIG. 6 may be executedby a processor of an electronic device that includes a fingerprintsensor.

With reference to FIG. 6, an input image 80 may be divided into aplurality of first regions 81, and a registered image 82 may be dividedinto a plurality of second regions 83. The input image 80 and theregistered image 82 may correspond to at least a portion of the user'sfingerprint, and the registered image 82 may be an image that haspreviously been stored before the input image 80 is generated. In anexample embodiment, the registered image 82 may include a plurality ofregistered component images, and in this case, the processor may divideeach registered image 82 into a plurality of second regions 83.

When the region division is completed, the processor may perform imageprocessing 91 for the first regions 81. The image processing 91 mayinclude arithmetic operations such as Fourier transform, log-polartransform, and the like. In an example embodiment, by respectivelyapplying a Fourier transform to the first regions 81, each magnitudecomponent of the first regions 81 may be computed, and by reapplying thelog-polar transform and the Fourier transform thereto, phase componentsof the first regions 81 may be calculated. Image processing 92 of thesecond regions 83 may also include operations which are similar tooperations of the image processing 91 of the first regions 81.

When the image processing 91, 92 for the first regions 81 and the secondregions 83 is completed, the processor may derive a phase correlation 93by using phase components of the first regions 81 and phase componentsof the second regions 83. The phase correlation 93 may be represented byan association between the first regions 81 and the second regions 83,based on a phase change defined by an angle.

The processor may define an angle at which a peak value is representedin the phase correlation 93, as a first correction value to be used forperforming a rotation transform of the registered image. In an exampleembodiment, when a peak value in the phase correlation 93 is 72 degrees,the processor may define the first correction value as 72 degrees, andmay perform a rotation transform 94 by rotating the registered image 82by 72 degrees.

When the rotation transform of the registered image 82 is completed, theprocessor may perform image processing 96 for the rotated registeredimage. In the example embodiment illustrated in FIG. 6, although theinput image 80 is also illustrated as being subjected to imageprocessing 95, in a manner different therefrom, the result of firstimage processing 91 executed after the region division may also be used.When the image processing operations 95 and 96 are completed, theprocessor may analyze a phase correlation 97 between the input image 80and the rotated registered image.

As a result of the phase correlation 97, the processor may calculate asecond correction value to be used for performing a translationtransform of the rotated registered image. The processor may allow atleast portions of the input image 80 and the registered image 82 tooverlap each other and thus derive an overlap region, by moving therotated registered image, based on the second correction value. As theoverlap region is derived between the input image 80 and the registeredimage 82, the image matching process may be completed.

FIG. 7 is a diagram illustrating log-polar conversion, according to anexample embodiment.

The log-polar conversion according to an example embodiment illustratedin FIG. 7 may be employed in at least one of the image processingoperations 91, 92, 95 and 96 of the image matching method describedabove with reference to FIG. 6. Referring to FIG. 7, concentric circlesmay be set with respect to a center point 1100 in an orthogonalCartesian coordinate system, and the concentric circles may be dividedinto a plurality of regions according to any one of a radius, an angle,a combination of a radius and an angle, and the like. The log-polartransformation may include a method in which the plurality of regions,defined on the orthogonal Cartesian coordinate system, are mapped tocorresponding regions on a polar coordinate system.

Referring to the polar coordinate system, the center point 1100 of theorthogonal Cartesian coordinate system may be mapped to coordinates (0,0°) of the polar coordinate system. In addition, first, second, third,and fourth regions 1110, 1120, 1130 and 1140 of the orthogonal Cartesiancoordinate system may be mapped to first, second, third, and fourthregions 1115, 1125, 1135 and 1145 of the polar coordinate system,respectively.

FIG. 8 is a view illustrating a method for processing fingerprintinformation, according to an example embodiment. In an exampleembodiment, the method for processing fingerprint information to bedescribed with reference to FIG. 8 may be executed in a processor of anelectronic device that includes a fingerprint sensor.

Referring to FIG. 8, the processor may compare an input image 100 withan original registered image 110 in order to determine whether a user'sfingerprint that has generated the input image is authenticated. Theinput image 100 may be divided into a plurality of first regions 101,102, and 103, and the original registered image 110 may be divided intoa plurality of second regions 111, 112, and 113. In the exampleembodiment illustrated in FIG. 8, at least a portion of the firstregions 101, 102, and 103 may overlap each other (i.e., a portion offirst region 101 may overlap a portion of first region 102, and aportion of first region 102 may overlap a portion of first region 103),and the overlapped regions may have the same area. Similarly thereto, atleast a portion of the second regions 111, 112, and 113 may overlap eachother (i.e., a portion of second region 111 may overlap a portion ofsecond region 112, and a portion of second region 112 may overlap aportion of second region 113), and the overlapped regions may have thesame area. In addition, in an example embodiment, each of the firstregions 101, 102, and 103 and the second regions 111, 112, and 113 mayalso have the same area.

The processor may convert pieces of information regarding time domainsincluded in the first regions 101, 102, and 103 and the second regions111, 112, and 113 into pieces of information 104, 105, and 106 and 114,115, and 116 regarding frequency domains, respectively, by using aFourier transform. The pieces of information 104, 105, and 106 and 114,115, and 116 regarding frequency domains, generated by the Fouriertransform, may be based on the orthogonal Cartesian coordinate systemrepresenting information as (x, y) coordinates. The pieces ofinformation regarding time domains included in the first regions 101,102, and 103 may be converted into the pieces of information 104, 105,and 106 regarding frequency domains, for example, pieces of firstinformation 104, 105, and 106, and the pieces of information regardingtime domains included in the second regions 111, 112, and 113 may beconverted into the pieces of information 114, 115, and 116 regardingfrequency domains, for example, pieces of second information 114, 115,and 116.

Subsequently, the processor may compare the pieces of first information104, 105, and 106 with the pieces of second information 114, 115, and116, and may select respective pieces of information having a highestcorrelation with each other, therefrom, as matching information.Referring to the example embodiment illustrated in FIG. 8, firstmatching information 105 may be selected from among the pieces of firstinformation 104, 105, and 106, and second matching information 116 maybe selected among pieces of second information 114, 115, and 116. Thefirst matching information 105 may correspond to one of the plurality offirst regions of the input image 100, for example, a first matchingregion 102, and the second matching information 116 may correspond toone of the plurality of second regions of the original registered image110, for example, a second matching region 113.

The processor may respectively apply the log-polar transformation to thepieces of first and second matching information 105 and 116, in order togenerate pieces of matching first and second polar coordinateinformation 107 and 117. The processor may generate first matching phaseinformation 108 and second matching phase information 118 from the firstand second matching polar coordinate information 107 and 117,respectively, by using a Fourier transform.

Then, the processor may perform a phase correlation by using the firstmatching phase information 108 and the second matching phase information118, thereby obtaining a graph as illustrated in the example embodimentof FIG. 8. Referring to the graph shown at a lower left portion of FIG.8, a peak value may be detected at an angle θ of 48 degrees, which mayindicate that when the original registered image 110 is rotated by 48degrees, the probability of matching between the original registeredimage 110 and the input image 100 is relatively highest. The processormay select the angle θ at which the peak value is detected, as a firstcorrection value.

With reference to the example embodiment illustrated in FIG. 8, theprocessor may rotate the original registered image 110 by using thefirst correction value, in order to thereby generate a rotatedregistered image 120. A reference point at the time of performing arotation transform of the original registered image 110 may be selectedvariably according to example embodiments, and the example embodimentillustrated in FIG. 8 may correspond to the case in which the rotationtransform is performed based on a center point of the originalregistered image 110.

The processor may respectively generate input phase information 109 andregistration phase information 129, by reapplying a Fourier transform tothe input image 100 and the rotated registered image 120. Pieces ofinformation included in the input phase information 109 and theregistration phase information 129 may be based on a Cartesiancoordinate system within which information is represented in (x, y)coordinates. Thus, the processor may generate a second correction valueto be used for performing a translation transform of the rotatedregistered image 120 by comparing the input phase information 109 andthe registration phase information 129 with each other. The processormay generate a reference registered image 130 for image matching bymoving the rotated registered image 120 based on the second correctionvalue.

With reference to FIG. 8, at least portions of the reference registeredimage 130 and the input image 100 may overlap with each other, and aregion in which the reference registered image 130 and the input image100 overlap each other may be defined as an overlap region 140. As anarea of the overlap region 140 is increased, or as a similarity betweenthe reference registered image 130 and the input image 100 in theoverlap region 140 is increased, it may be determined that the inputimage 100 and the original registered image 110 are sufficiently similarto each other. For example, the processor may determine whether theinput image 100 is authenticated, based on at least one of an area ofthe overlap region 140, and a matching score calculated in the overlapregion 140. In an exemplary embodiment, the matching score may becalculated by performing a normalized cross-correlation (NCC).

FIG. 9 is a flow chart illustrating a method for processing fingerprintinformation, according to an example embodiment. In an exampleembodiment, a method for processing fingerprint information to bedescribed with reference to FIG. 9 may be executed in a processor of anelectronic device that includes a fingerprint sensor.

Referring to FIG. 9, a method for processing fingerprint informationaccording to an example embodiment may begin with receiving an inputimage that corresponds to at least a portion of a user's fingerprint, inoperation S1010. For example, when the input image is received, aprocessor may divide the input image into a plurality of first regions,and the processor may also divide a registered image into a plurality ofsecond regions in operation S1011, and may compare the first regions andthe second regions with each other in operation S1012. In this case,before comparing the first and second regions with each other, pieces ofinformation regarding a time domain included in the first and secondregions may be converted into pieces of information regarding afrequency domain in order to reduce a required amount of computation.

The processor may select a first matching region from the first regionsand a second matching region from the second regions, based on acomparison result of the first regions and the second regions, inoperation S1013. The first matching region and the second matchingregion may be respective regions determined to have a highestprobability of matching from among the first regions and the secondregions. The processor may generate a first correction value to be usedfor performing a rotation transform, by using the first matching regionand the second matching region, in operation S1014.

The first correction value may be obtained by analyzing a phasecorrelation between the first matching region and the second matchingregion, and the second matching region may be rotated according to thefirst correction value in operation S1015. The processor may rotate theentirety of a registered image based on the first correction value, andthus, the second matching region may be rotated together therewith.

The processor may generate a second correction value by comparing thesecond matching region of the rotated registered image with the firstmatching region in operation S1016. The second correction value may be avalue that corresponds to an amount of shift of the rotated registeredimage in order to be matched to the input image. In an exampleembodiment, the second correction value may be determined variably basedon reference point coordinates of the rotation transform performed inoperation S1015 in which the registered image is rotated and transformedby using the first correction value.

For example, when the second correction value is determined, theprocessor may shift the registered image according to the secondcorrection value to match the registered image and the input image inoperation S1017, and may determine whether the user fingerprint isauthenticated therefrom in operation S1018. The authentication may bedetermined by using any of an area of an overlap region formed byoverlapping the matched registered image and the input image, a matchingscore calculated in the overlap region, and the like.

FIGS. 10 and 11 are views illustrating a method for processingfingerprint information, according to an example embodiment. In anexample embodiment, a method for processing fingerprint information tobe described with reference to FIGS. 10 and 11 may be executed in aprocessor of an electronic device that includes a fingerprint sensor.

With reference to FIG. 10, a processor may divide an input image 200into a plurality of first regions 201, 202, and 203, and the processormay divide an original registered image 210 into a plurality of secondregions 211, 212, and 213. In an example embodiment, each of the firstregions 201, 202, and 203 and each of the second regions 211, 212, and213 may also have the same area. In the example embodiment illustratedin FIG. 10, although the number of the first regions 201, 202, and 203included in a single input image 200 and the number of the secondregions 211, 212, and 213 included in a single original registered image210 are assumed to be equal to each other, in a manner differenttherefrom, the number of the first regions 201, 202, and 203 may also bedifferent from the number of the second regions 211, 212, and 213.

The processor may convert pieces of information regarding time domainsincluded in the first regions 201, 202, and 203 and the second regions211, 212, and 213 into pieces of information 204, 205, and 206 and 214,215, and 216 regarding frequency domains, respectively, by using aFourier transform. The processor may compare the pieces of information204, 205, and 206 and 214, 215, and 216, for example, the pieces offirst information 204, 205, and 206 with the pieces of secondinformation 214, 215, and 216 in frequency domains, respectively, andmay thus select respective pieces of information having a highestcorrelation with respect to each other, as matching information.Referring to the example embodiment illustrated in FIG. 10, firstmatching information 205 may be selected from the pieces of firstinformation 204, 205, and 206, and second matching information 216 maybe selected from the pieces of second information 214, 215, and 216. Thefirst matching information 205 may correspond to a first matching region202 of the input image 200, and the second matching information 216 maycorrespond to a second matching region 213 of the original registeredimage 210.

The processor may generate pieces of matching first and second polarcoordinate information 207 and 217 by applying a log-polartransformation to each of the pieces of first and second matchinginformation 205 and 216, and may generate first matching phaseinformation 208 and second matching phase information 218 from thepieces of matching first and second polar coordinate information 207 and217, respectively, by using a Fourier transform. Then, the processor maycalculate a phase correlation between the pieces of first and secondmatching phase information 208 and 218, and thus, may calculate a firstcorrection value to be used for performing a rotation transform of theoriginal registered image 210.

Referring to a graph of the phase correlation illustrated in a lowerleft portion of FIG. 10, a peak value may be detected at an angle θ of48 degrees, which indicates that when the original registered image 210is rotated by 48 degrees, a probability of matching between the originalregistered image 210 and the input image 200 is relatively highest. Theprocessor may select the angle θ at which the peak value is detected, asthe first correction value.

With reference to the example embodiment illustrated in FIG. 10, theprocessor may rotate the original registered image 210 by using thefirst correction value, thereby generating a rotated registered image220. A reference point at the time of performing the rotation transformof the original registered image 210 may be selected variably accordingto an example embodiment. The processor may respectively generate inputphase information 209 and registration phase information 221, byreapplying a Fourier transform to the input image 200 and the rotatedregistered image 220.

A second matching region 223 of the rotated registered image 220 maycorrespond to the second matching region 213 of the original registeredimage 210. In the example embodiment illustrated in FIG. 10, theprocessor may set a conversion target region 224 that includes thesecond matching region 223 in the rotated registered image 220, and mayapply a Fourier transform to each of the conversion target region 224and the first matching region 202. Thus, an amount of calculation may bereduced as compared with the case in which the Fourier transform isapplied to the entirety of the input image 100 and the rotatedregistered image 120 as in the example embodiment of FIG. 8. In theexample embodiment illustrated in FIG. 10, the input image 200 and therotated registered image 220 may each have a resolution of 128×128, andthe first matching region 202 and the conversion target region 224 mayeach have a resolution of 64×64. Thus, by only applying a Fouriertransform to the first matching region 202 and the conversion targetregion 224, the amount of required calculation may be significantlyreduced.

The processor may calculate a second correction value to be used forperforming a translation transform by comparing the input phaseinformation 209 and the registration phase information 229 generated viathe Fourier transform with each other. A reference registered image 230may be generated by moving the rotated registered image 220 based on thesecond correction value, and the reference registered image 230 mayoverlap the input image 200 to provide an overlap region 240. Theprocessor may determine whether or not the input image 200 isauthenticated, based on at least one of an area of the overlap region240, and a matching score calculated with respect to the overlap region240.

Conversely, the second correction value may be determined variably basedon a position in which the reference point is set when the originalregistered image is rotated based on the first correction value.Referring to FIG. 11, a second matching region 301 may be selected in anoriginal registered image 300, and a first correction value may becalculated by comparing the selected second matching region with a firstmatching region of an input image. For example, when the originalregistered image 300 is rotated by using the first correction value, asecond correction value in a case in which the original registered image300 is rotated based on a center of the original registered image 300and a second correction value in a case in which the original registeredimage 300 is rotated based on a center of the second matching region 301may be calculated differently.

With reference to FIG. 11, for example, when the rotation transform isperformed based on a center of the original registered image 300, asecond matching region 311 and a conversion target region 312 may bedisposed at a right lower end of a rotated registered image 310. In thiscase, the calculated second correction value may be (t_(x1), t_(y1)).Conversely, for example, when the rotation transform is performed basedon a center of the second matching region 301, a second matching region321 and a conversion target region 322 may be disposed at a right upperend of a rotated registered image 320. In this case, the calculatedsecond correction value may be (t_(x2), t_(y2)), which may be a valuethat is different from the corresponding value (t_(x1), t_(y1)) of theforegoing case described above.

FIG. 12 is a diagram illustrating a user authentication method,according to an example embodiment. In an example embodiment, the userauthentication method to be described with reference to FIG. 12 may beexecuted in a processor of an electronic device that includes afingerprint sensor.

In the example embodiment illustrated in FIG. 12, an input image 400 maybe compared with each of a plurality of registered images. The number ofregistered images may vary, depending on an example embodiment, and aprocessor may compute a matching probability between each of theregistered images and the input image 400 by comparing the input image400 with the registered images. The matching probability may becalculated from a phase correlation between first regions generated bydividing the input image 400 and second regions generated by dividingeach of the registered images. In an example embodiment, the processormay compare a peak value appearing in phase correlations between thefirst and second regions with a predetermined threshold value, and mayselect only registered images that have a phase correlation with a peakvalue which is greater than the threshold value, as a candidate group.

The candidate group selection may be derived from a phase correlationwhich is performed in order to obtain a first correction value to beused for performing a rotation transform. Referring to a thirdregistered image 410 in the example embodiment illustrated in FIG. 12,the third registered image 410 may be divided into three second regions411, 412 and 413, and a phase correlation between each of the secondregions 411, 412 and 413 and a first region 401 of the input image 400may be obtained. As a result of the phase correlation analysis, forexample, when the peak value of the phase correlation between the secondregion 412 and the first region 401 of the input image 400 has a highestvalue of 0.81, a matching probability of the third registered image 410may be determined to be 0.81. For example, the matching probability ofeach of the registered images may be defined by a highest peak valueacquired from the phase correlation obtained by comparing the secondregions obtained by dividing each of the registered images with thefirst regions of the input image.

The processor may determine whether a user's fingerprint isauthenticated by matching the input image 400 with each of candidateregistered images 410, 420 and 430 included in the candidate group. Inthis aspect, in order to significantly reduce an amount of calculationto match the input image 400 with the candidate registered images 410,420 and 430, the processor may sort the candidate registered images 410,420 and 430 in a sequential order that corresponds to the matchingprobability. Referring to FIG. 12, starting with the third registeredimage 410 which has a highest matching probability (e.g., 0.81), thesixth registered image 420 (which has a matching probability of 0.75)and the first registered image 430 (which has a matching probability of0.73) may be sequentially matched with the input image 410.

The matching of the third registered image 410 and the input image 400may be performed by rotating and moving the third registered image 410to be converted into a third reference registered image 415. Theprocessor may calculate an area of an overlap region 405 in which thethird reference registered image 415 and the input image 400 overlapeach other and a matching score with respect to the overlap region 405.For example, when the area of the overlap region 405 and the matchingscore with respect to the overlap region 405 do not satisfypredetermined criteria, the processor may match the input image 400 withthe sixth registered image 420 and then may determine whether the inputimage 400 is authenticated. If the input image 400 is not authenticatedfor any of the candidate registered images 410, 420 and 430 included inthe candidate group in a manner similar thereto as described above, theprocessor may ultimately determine that the authentication is to berejected. Meanwhile, when the input image 400 is authenticated for anyone of the candidate registered images 410, 420 and 430 included in thecandidate group, the processor may provide an authentication permissionjudgment for the input image 400, regardless of an authentication resultfor the other subordinate registered images.

FIGS. 13, 14A, 14B, 14C, and 14D are views illustrating a method forprocessing fingerprint information, according to an example embodiment.In an example embodiment, the method for processing fingerprintinformation to be described with reference to FIGS. 13, 14A, 14B, 14C,and 14D may be executed in a processor of an electronic device thatincludes a fingerprint sensor.

Referring to FIG. 13, a fingerprint image 500, which may become an inputimage or a registered image, may be divided into a plurality ofsub-regions. The sub-regions may include a first sub-region 501, asecond sub-region 502 and a third sub-region 503. The first sub-region501 may be a region that includes minutiae which appear in a fingerprintof a person, and the second sub-region 502 may be a region that isadjacent to the first sub-region 501. The third sub-region 503 may bedefined as a region other than the first and second sub-regions 501 and502.

As in the example embodiment illustrated in FIG. 13, the fingerprintimage 500 may be divided into a plurality of sub-regions, and differentrespective weights may be assigned to each of the first, second, andthird sub-regions 501, 502, and 503, to be used for authentication of auser fingerprint. In this case, a security problem, in which an inputimage generated from a fingerprint of a user not having userauthorization for a device may accidentally overcome the registeredimage, may occur.

FIGS. 14A, 14B, 14C, and 14D are diagrams illustrating examples ofdistribution of various types of minutiae represented in a fingerprint.First, referring to FIG. 14A, a total of three unique regions 601 a thatinclude minutiae may be included in a fingerprint image 600 a. However,an area of the unique region 601 a that has the minutiae may be smallerthan that of a general region 602 a. A weight assigned to the uniqueregion 601 a may be higher than a weight assigned to the general region602 a. In an example embodiment, even when a user's fingerprint that hasa region that coincides with the general region 602 a excluding theunique regions 601 a is input, user authentication may be denied unlessa minutiae is detected commonly in the unique regions 601 a.

In FIGS. 14B, 14C, and 14D, a user authentication method which issimilar to that in the foregoing example embodiment described withreference to FIG. 14A may also be applied. Referring to FIG. 14B, twounique regions 601 b may be present in a fingerprint image 600 b, and ageneral region 602 b may have a much larger area than areas of the twounique regions 601 b. For example, in a case in which a user'sfingerprint that coincides with the general region 602 b is input, in anexisting user authentication method, an overlap region may besufficiently secured and a normalized cross-correlation in the overlapregion is also calculated to have a relatively high value, and thus,user authentication may be allowed. However, in an example embodiment,as a weight is assigned to the two unique regions 601 b, in a case inwhich only the general region 602 b coincides with a user's fingerprint,the user authentication may be rejected.

In an example embodiment illustrated in FIG. 14C, a weight may also beassigned to three feature points 601 c, and user authentication may notbe permitted when only a general region 602 c coincides therewith. In afingerprint image 600 d illustrated in FIG. 14D, a unique region thathas minutiae may not be present. Thus, in a case in which thefingerprint image 600 d illustrated in FIG. 14D is input, userauthentication may not be permitted.

FIGS. 15 and 16 are views illustrating a user authentication method,according to an example embodiment. In an example embodiment, a userauthentication method to be described with reference to FIGS. 15 and 16may be executed in a processor of an electronic device that includes afingerprint sensor.

Referring first to FIG. 15, an input image 700 and a referenceregistered image 710 that have been subjected to a rotation transformand a translation transform for image matching are illustrated. Each ofthe input image 700 and the reference registered image 710 may bedivided into a plurality of sub-regions, and the plurality ofsub-regions may be weighted according to whether minutiae are retained.In an example embodiment, a respective weight may be assigned to each ofa first sub-region 701 included in the input image 700 and a firstsub-region 711 included in the reference registered image 710.Conversely, a weight may not be assigned to each of a second sub-region702 included in the input image 700 and a second sub-region 712 includedin the reference registered image 710.

The reference registered image 710 and the input image 700 may bematched to form an overlap region 720. A processor may determine whetherto authenticate the input image 700, based on at least one of an area ofthe overlap region 720 and a matching score calculated with respect tothe overlap region 720. In the example embodiment illustrated in FIG.15, the processor may assign a weight to a first sub-region 721 locatedin the overlap region 720, but may not assign a weight to a secondsub-region 722. Thus, as the overlap region 720 has an increased amountof minutiae, a matching score of the input image 700 may increase and anauthentication success probability may be increased.

Next, referring to FIG. 16, an input image 730, and a referenceregistered image 740 that have been subjected to a rotation transformand a translation transform for image matching are illustrated. Each ofthe input image 730 and the reference registered image 740 may bedivided into a plurality of sub-regions 732 and 742, and the pluralityof sub-regions may be weighted according to whether minutiae areretained therein. In the example embodiment illustrated in FIG. 16,minutiae may not be present in each of sub-regions 752 of an overlapregion 750 in which the input image 730 and the reference registeredimage 740 are matched with each other. Thus, in the example embodimentillustrated in FIG. 16, a matching score in the overlap region 750 maybe relatively low, and thus, user authentication of the input image 730may be denied.

For example, when determining whether to authenticate the input image730 by using only an area of the overlap region 750 and a matching scorewithin the overlap region 750, authentication of the input image 730 mayalso be allowed. However, in the example embodiment, in a process ofcalculating the matching score in the overlap region 750, a weight maybe assigned to a sub-region that has minutiae. Thus, in the exampleembodiment of FIG. 16 in which no minutiae are included in the overlapregion 750, the authentication of the input image 730 may be rejected,and security performance may be improved.

FIGS. 17 and 18 are views illustrating a method for processingfingerprint information, according to an example embodiment. In anexample embodiment, a method for processing fingerprint information tobe described with reference to FIGS. 17 and 18 may be executed in aprocessor of an electronic device that includes a fingerprint sensor.

Referring to FIG. 17, a processor may compare an input image 800 with aregistered image 810 in order to determine whether a user fingerprintthat has generated the input image is authenticated. In this case, boththe input image 800 and the registered image 810 do not includeinformation regarding an image captured from a shape of an intactfingerprint, but may only include information regarding a feature pointpresent in the fingerprint. For example, the feature point may includeminutiae of the fingerprint. As both the input image 800 and theregistered image 810 do not store the imaged information, securityperformance may be further enhanced. Since both the input image 800 andthe registered image 810 include only information regarding a featurepoint, the influence of noise components that might otherwise occur in aphase correlation analysis may be reduced, and thus, a process ofdividing the input image 800 and the registered image 810 into aplurality of regions may also be omitted.

Matching and authentication processes of the input image 800 and theregistered image 810 may be similar to those in the foregoing exampleembodiments described above. Pieces of frequency domain information 805and 815 may be generated by applying a Fourier transform to the inputimage 800 and the registered image 810, and pieces of polar coordinateinformation 806 and 816 may be generated by applying a log-polartransform to the pieces of frequency domain information 805 and 815.Subsequently, pieces of phase information 807 and 817 may be generatedby applying the Fourier transform to the pieces of polar coordinateinformation 806 and 816, and a phase correlation between the pieces ofphase information 807 and 817 may be analyzed to calculate a firstcorrection value. In the example embodiment illustrated in FIG. 17, thefirst correction value may be 37 degrees, as indicated in the graphprovided at a lower left portion of FIG. 17.

A rotated registered image 820 may be generated by rotating theregistered image 810 based on the first correction value, and inputphase information 809 and registration phase information 821 may begenerated by applying a Fourier transform to each of the input image 800and the rotated registered image 820. The processor may compare theinput phase information 809 with the registration phase information 821in order to calculate a second correction value to be used forperforming a translation transform of the rotated registered image 820,and may generate a reference registered image 830 by moving the rotatedregistered image 820 according to the second correction value.

The reference registered image 830 may be matched with the input image800, to form an overlap region 840. Similar to the registered image 810,the reference registered image 830 also includes only the featurepoints. In the example embodiment illustrated in FIG. 17, a featurepoint 801 of the input image 800 and a feature point 832 of thereference registered image 830 may coincide with each other in theoverlap region 840. Thus, a matching score within the overlap region 840may be calculated to be sufficiently high to indicate that thefingerprint is authentic, and authentication of the input image 800 maybe successful.

Next, referring to FIG. 18, an input image 900 and an originalregistered image 910 may have unique regions 901 and 911 which haverespective feature points. In addition, due to foreign substancespresent on a fingerprint sensor, a user fingerprint skin, or the like,error regions 902 and 912 in which a fingerprint is not properly imagedmay be present. Hereinafter, for convenience of explanation, it isassumed that the input image 900 is a legitimate user's fingerprint.

A process of matching the original registered image 910 may be similarto that in the foregoing example embodiment described with reference toFIG. 17. Pieces of frequency domain information 903 and 913 may begenerated by applying a Fourier transform to the input image 900 and theregistered image 910, and pieces of polar coordinate information 904 and914 may be generated by applying a log-polar transform to the pieces offrequency domain information 903 and 913. Subsequently, pieces of phaseinformation 905 and 915 may be generated by applying a Fourier transformto pieces of polar coordinate information 904 and 914, and a phasecorrelation between the pieces of phase information 905 and 915 may beanalyzed to calculate a first correction value to be used for performingrotation transform. In the example embodiment illustrated in FIG. 18,the first correction value may be 37 degrees, as indicated in the graphprovided at a lower left portion of FIG. 18.

A rotated registered image 920 may be generated by rotating the originalregistered image 910 based on the first correction value, and inputphase information 906 and registration phase information 926 may begenerated by applying a Fourier transform to each of the input image 900and the rotated registered image 920. The processor may compare theinput phase information 906 with the registration phase information 926in order to calculate a second correction value to be used forperforming a translation transform of the rotated registered image 920,and may generate a reference registered image 930 by moving the rotatedregistered image 920 according to the second correction value.

At least portions of the reference registered image 930 and the inputimage 900 may overlap each other to form an overlap region 940. In thiscase, since a plurality of error regions may be present in the overlapregion 940, a matching score may be calculated to be relatively low dueto the error regions, and thus, authentication of the input image 900may fail. In order to prevent the problems as described above, in theexample embodiment illustrated in FIG. 18, a weight may be assigned to aunique region 941 having a feature point in the overlap region 940.Thus, despite the presence of an error region in the overlap region 940,authentication of the input image 900 may be allowed due to the weightassigned to the unique region 941, and the performance of fingerprintrecognition may be improved.

FIG. 19 is a diagram illustrating a registered image processing method,according to an example embodiment. In an example embodiment, a methodfor processing fingerprint information to be described with reference toFIG. 19 may be executed in a processor of an electronic device thatincludes a fingerprint sensor.

Referring to FIG. 19, an input image 1201 acquired from a userfingerprint may be compared with a plurality of registered images 1211,1212, and 1213 stored in a memory 1210. In an example embodiment of FIG.19, a third registered image 1213 and the input image 1201 may have arelatively high similarity, and a processor may rotate and/or move thethird registered image 1213 in order to generate a reference registeredimage 1220. A reference registered image 1220 may be an image that isable to be matched with the input image 1201. The processor maydetermine whether the input image 1201 is authenticated, based on atleast one of an area of an overlap region 1230 formed by overlapping thereference registered image 1220 and the input image 1201, and a matchingscore calculated with respect to the overlap region 1230.

For example, when the input image 1201 and the third registered image1213 are compared with each other and authentication of the input image1201 is successful, the processor may determine a similarity between theinput image 1201 and the third registered image 1213 in order todetermine whether a template formed by combining the input image 1201and the third registered image 1213 is generated. The template may haveat least one fingerprint image, and may function as reference data to becompared with the input image input via a fingerprint sensor in order todetermine whether the user fingerprint is authenticated. For example, inan example embodiment, for example, when a specific condition issatisfied, the input image 1201 that has succeeded in the userauthentication may be merged with at least one of the registered images1211, 1212, and 1213 that have previously been stored, to thus bemanaged as a single template.

In the example embodiment illustrated in FIG. 19, for example, when theauthentication of the user fingerprint is determined to be successful asa result of comparing the input image 1201 with the third registeredimage 1213, the processor may determine whether to merge the input image1201 and the third registered image 1213 and to manage the merged imagesas a single template. In this case, reference similarity in merging theinput image 1201 and the third registered image 1213 into a singletemplate may have a higher value than that of similarity between theinput image 1201 and the third registered image 1213, thereby becomingan authentication reference of the user fingerprint.

FIGS. 20 and 21 are flow charts illustrating a registered imageprocessing method, according to an example embodiment. In an exampleembodiment, the registered image processing method to be described withreference to FIGS. 20 and 21 may be executed in a processor of anelectronic device that includes a fingerprint sensor.

Referring to FIG. 20, the registered image processing method accordingto an example embodiment may begin with receiving a partial fingerprintimage that corresponds to at least a portion of a user fingerprint, in auser fingerprint registration operation S1020. The partial fingerprintimage may be acquired from a portion of the user fingerprint that is incontact with a fingerprint sensor, and may be provided to obtain userregistration. A processor may determine whether the partial fingerprintimage acquired in operation S1020 is a first input image, in operationS1021. For example, if there is no fingerprint image received before thepartial fingerprint image acquired in operation S1020 and storedpreviously, the partial fingerprint image acquired in operation S1020may be determined to be a first input image.

When it is determined in operation S1021 that the partial fingerprintimage is the first input image, the processor may generate a newtemplate in operation S1024, and may store the partial fingerprint imagein the generated new template, in operation S1025. The partialfingerprint image stored in the template may be used as a registeredimage in a subsequent user authentication process.

As a result of the determination in operation S1021, for example, whenit is determined that the partial fingerprint image is not the firstinput image, the processor may compare the partial fingerprint imagewith previously acquired and stored templates in operation S1022. Thecomparison in operation S1022 may be performed by comparing the partialfingerprint image with each of the registered images stored in thetemplates. In an example embodiment, the processor may compare thepartial fingerprint image with each of the registered images included inthe plurality of templates, respectively, in order to determine whethera similar template able to be matched with the partial fingerprint imageis present in operation S1023.

As a result of the determination in operation S1023, if there is nosimilar template that is able to be matched with the partial fingerprintimage, the processor may generate a new template in operation S1024, andmay store the partial fingerprint image in the generated new template,in operation S1025. Conversely, as the result of the determination inoperation S1023, when the similar template that is able to be matchedwith the partial fingerprint image is present, the processor may storethe partial fingerprint image in the similar template to update thesimilar template in operation S1026. For example, in a case in which thesimilar template retrieved in operation S1023 has two existingregistered images, the partial fingerprint image may be added inoperation S1026, such that the similar template may include a total ofthree registered images.

When the template update is completed, the processor may compare each ofthe templates with other templates in operation S1027. For example, whenfirst, second, and third templates are present, the first and secondtemplates may be compared with each other, the second and thirdtemplates may be compared with each other, and the third and firsttemplates may be compared with each other. The comparison in operationS1027 may include an image comparison process, and the processor maydetermine whether templates able to be merged with each other arepresent by determining a similarity of images provided by the templates,in operation S1028.

In a case in which a template that is able to be merged is present inoperation S1028, the processor may combine the templates that are ableto be merged with each other into a single template in operation S1029.For example, when the first and third templates have relatively highsimilarity and are thus merged into a fourth template, the processor mayadd an authentication success count of the first template and anauthentication success count of the third template, and thus, maycalculate an authentication success count of the fourth template. Aftercompleting the template merging, the processor may arrange and managethe respective templates according to the number of authenticationsuccesses, in operation S1030. If there is no template that is able tobe merged in operation S1028, the processor may arrange and manage thetemplates according to the number of authentication successes withoutmerging the templates in operation S1030.

For example, in the example embodiment described with reference to FIG.20, when a partial fingerprint image for user registration is received,the partial fingerprint image may be compared with previously registeredtemplates in order to determine whether a similar template with arelatively high similarity therewith is present. When the similartemplate that has a relatively high similarity with the partialfingerprint image is present, the partial fingerprint image may bemerged with the similar template to be stored therein, while all thetemplates including the similar template may be compared with each otherto determine a possibility of merging at least portions of thetemplates.

Next, referring to FIG. 21, a registered image processing methodaccording to an example embodiment may begin with receiving an inputimage that corresponds to at least a portion of a user fingerprint, inoperation S1040. The input image may be acquired from a portion of theuser fingerprint that is in contact with a fingerprint sensor, and maybe provided to obtain user authentication. A processor may compare theinput image acquired in operation S1040 with templates that haveregistered images which have previously been stored, in operation S1041.Operation S1041 may be performed by image-matching and comparing theinput image with a registered image of the template.

When the image matching and comparison between the input image and thetemplate is completed, the processor may determine whether anauthentication template that is able to be matched with the input imageis present among the templates in operation S1042. If the authenticationtemplate is not present, the processor may determine that the inputimage has been generated from an illegitimate user, and may reject theuser authentication in operation S1043. Conversely, when theauthentication template is present, the processor may permit the userauthentication in operation S1044, in order to cancel a lock mode of anelectronic device, perform a payment process in a payment service, orthe like.

After allowing the user authentication, the processor may update theauthentication template by adding an input image to the authenticationtemplate in operation S1045. The processor may perform the updateoperation in operation S1045 by adding the input image to theauthentication template only when the similarity between the input imageand the authentication template is relatively high. The degree ofsimilarity of the authentication template with the input image requiredto update the authentication template may have a stricter criterion thanthat of a degree of similarity of the authentication template with theinput image required for user authentication permission.

When the template update is completed, the processor may perform imagecomparison between the templates in operation S1046. The processor maydetermine whether templates that are able to be merged are present inoperation S1047, from a result of the image comparison in operationS1046 in which the templates that have similarity are present. Forexample, when templates that are able to be merged are present, theprocessor may merge the templates able to be merged into a singletemplate in operation S1048, and may arrange and manage the templates inoperation S1049. The templates may be sorted according to any of thenumber of authentication successes in user authentication, the number ofregistered images included in each of the templates, a total area of theregistered images included in each of the templates, and the like. Ifthere is no template that is able to be merged as a result of thedetermination in operation S1047, the processor may arrange and managethe templates while omitting the template merging process in operationS1049.

According to an example embodiment, for example, when a similaritybetween an input image acquired from a fingerprint sensor and anauthentication template, one of a plurality of templates, is recognized,a processor may permit user authentication and may add the input imageto the authentication template, thereby updating the authenticationtemplate. For example, the input image that has been determined as animage input by a legitimate user may be added to the template recognizedas being similar to the input image, in order to continuously update thecorresponding template. Thus, accuracy of the fingerprint authenticationwith respect to various input conditions and environments may beimproved.

FIGS. 22A, 22B, and 22C are diagrams illustrating a registered imageprocessing method, according to an example embodiment. In an exampleembodiment, the registered image processing method to be described withreference to FIGS. 22A, 22B, and 22C may be executed in a processor ofan electronic device that includes a fingerprint sensor.

First, referring to an example embodiment illustrated in FIG. 22A, atemplate 2000 may include a plurality of registered images. A processormay compare the template 2000 with an input image 2010 in order todetermine whether a user is authenticated. With reference to the exampleembodiment illustrated in FIG. 22A, the input image 2010 and registeredimages included in the template 2000 may form an overlap region, anduser authentication based on the input image 2010 may be successfulbased on an area of the overlap region, and a matching score between theinput image 2010 and the registered images. In addition, the input image2010 may be merged into the template 2000, to thereby be updated with anew template 2100.

Next, referring to an example embodiment illustrated in FIG. 22B, anarea of an overlap region formed between registered images included inthe template 2000 and an input image 2020 may be relatively very small.Thus, user authentication based on the input image 2020 may be denied,and in this case, the template 2000 may not be updated by the inputimage 2020.

Referring to an example embodiment illustrated in FIG. 22C, an overlapregion may be formed between registered images included in the template2000 and an input image 2030, and thus, user authentication based on theinput image 2030 may be successful. Compared with the case of FIG. 22A,however, the area of the overlap region formed in the example embodimentillustrated in FIG. 22C may be relatively small. In updating thetemplate 2000 using the input image 2030, a stricter criterion may beemployed than in the user authentication operation.

Thus, in the example embodiment illustrated in FIG. 22C, the userauthentication based on the input image 2030 has succeeded, while theupdate of the template 2000 using the input image 2030 may be rejected.For example, by setting criteria applied to the user authentication andthe template update differently from each other and applying arelatively strict criterion to the template update, an improperfingerprint image may be prevented from being added to the template, andthus, the template may be prevented from being contaminated. Inaddition, deterioration in reliability of the template may be prevented.

FIG. 23 is a block diagram of an electronic device, according to anexample embodiment.

Referring to FIG. 23, a fingerprint sensor 3010 according to an exampleembodiment may be applied to a computer device 3000. The computer device3000 according to the example embodiment illustrated in FIG. 23 mayinclude an input/output device 3020, a memory 3030, a processor 3040, aport 3050, and the like, as well as the fingerprint sensor 3010. Inaddition, the computer device 3000 may further include a wired/wirelesscommunications device, a power supply device, and the like. Amongconstituent elements illustrated in FIG. 23, the port 3050 may be adevice provided to allow the computer device 3000 to communicate with avideo card, a sound card, a memory card, a USB device, and the like. Thecomputer device 3000 may be implemented as a comprehensive conceptincluding any of a smartphone, a tablet PC, and a smart wearable device,and the like, as well as a general desktop computer and a laptopcomputer.

The processor 3040 may be configured to perform specific arithmeticoperations, commands, tasks, and the like. The processor 3040 may beimplemented as a central processing unit (CPU) or a microprocessor unit(MCU), and may be configured to communicate with the memory device 3030,the input/output device 3020, the fingerprint sensor 3010, and otherdevices connected to the port 3050, via a bus 3060.

The memory 3030 may include a storage medium that is configured to storedata required for operations of the computer device 3000, multimediadata, or the like. The memory 3030 may include a volatile memory, suchas a random access memory (RAM), or a non-volatile memory, such as aflash memory and the like. In addition, the memory 3030 may include atleast one of a solid state drive (SSD), a hard disk drive (HDD), and anoptical drive (ODD), as a storage device. The memory 3030 may store aregistered image to be compared with an input image input through thefingerprint sensor 3010 therein. The input/output device 3020 mayinclude an input device such as any of a keyboard, a mouse, atouchscreen, and the like for a user, and an output device such as anyof a display, an audio output unit, and the like.

The fingerprint sensor 3010 may be connected to the processor 3040 bythe bus 3060 or another communication means. The processor 3040 mayperform user authentication by comparing an input image received via thefingerprint sensor 3010 with a registered image stored in the memory3030. The user authentication process performed by the processor 3040may be performed according to various example embodiments as describedabove with reference to FIGS. 1 to 22.

As set forth above, according to various example embodiments, bydividing each of an input image and a registered image into a pluralityof regions and comparing the plurality of regions with each other tomatch the input image and the registered image, the accuracy offingerprint recognition may be increased. Further, by updating atemplate having a registered image by using an input image that has beenused successfully in performing an authentication, pieces of accuratefingerprint information may be accumulated to improve fingerprintrecognition performance.

While example embodiments have been shown and described above, it willbe apparent to those having ordinary skill in the art that modificationsand variations could be made without departing from the scope of thepresent inventive concept as defined by the appended claims.

1. A method of authenticating a user based on fingerprint recognition,comprising: receiving, by a fingerprint sensor, a fingerprint of theuser; dividing, by a processor, an input image that corresponds to atleast a portion of the fingerprint of the user into a plurality of firstregions; dividing, by the processor, a registered image that haspreviously been stored into a plurality of second regions; selecting, bythe processor, a first matching region from among the plurality of firstregions and selecting a second matching region from among the pluralityof second regions, based on a comparison between the plurality of firstregions and the plurality of second regions; matching, by the processor,the registered image with the input image by comparing the firstmatching region with the second matching region; and authenticating theuser based on a result of the matching.
 2. The method of claim 1,further comprising: determining, by the processor, a plurality ofmatching probabilities that correspond to similarities between each ofthe plurality of first regions and each of the plurality of secondregions; and determining, by the processor, the first matching regionand the second matching region based on a highest matching probabilityfrom among the plurality of matching probabilities.
 3. The method ofclaim 2, wherein the determining the plurality of matching probabilitiescomprises determining the plurality of matching probabilities bydetermining phase correlations between each of the plurality of firstregions and each of the plurality of second regions.
 4. (canceled) 5.The method of claim 3, wherein a peak value of each of the phasecorrelations is detected to generate correction values for performing arotation transform of the registered image.
 6. The method of claim 1,wherein the matching the registered image with the input imagecomprises: rotating the registered image, based on a first correctionvalue generated by comparing the first matching region with the secondmatching region; and matching the registered image with the input imageby moving the registered image that has been rotated based on a secondcorrection value, wherein the second correction value is generated bycomparing the input image with the registered image that has beenrotated.
 7. The method of claim 6, wherein the second correction valueis generated by comparing the first matching region with the secondmatching region after the registered image has been rotated based on thefirst correction value.
 8. (canceled)
 9. The method of claim 1, furthercomprising: determining, by the processor, a matching score thatindicates a similarity between the registered image and the input image,in an overlap region formed by the matching the registered image withthe input image; determining, by the processor, an area of the overlapregion; and determining, by the processor, whether the fingerprint isauthenticated, based on at least one from among the matching score andthe area of the overlap region.
 10. (canceled)
 11. The method of claim9, wherein the determining the matching score comprises: dividing theoverlap region into a plurality of sub-regions; searching minutiae inthe overlap region; and determining a normalized cross-correlationbetween the registered image and the input image in the overlap regionby assigning a predetermined weight to each of a first sub-region thatincludes the minutiae and a second sub-region adjacent to the firstsub-region. 12-14. (canceled)
 15. A method of authenticating a userbased on fingerprint recognition, comprising: receiving, by afingerprint sensor, a fingerprint of the user; dividing, by a processor,each of an input image that corresponds to at least a portion of thefingerprint of the user and a registered image that has previously beenstored into a plurality of sub-regions; finding, by the processor, fromamong the plurality of sub-regions of each of the registered image andthe input image, a first sub-region that is included in an overlapregion in which the registered image overlaps the input image and asecond sub-region that is included in the overlap region and is adjacentto the first sub-region, each of the first sub-region and the secondsub-region including minutiae; and determining, by the processor,whether the user is authenticated, by assigning a first predeterminedweight to the first sub-region and a second predetermined weight to thesecond sub-region.
 16. The method of claim 15, wherein the firstpredetermined weight assigned to the first sub-region is greater thanthe second predetermined weight assigned to the second sub-region. 17.The method of claim 15, wherein the determining whether the fingerprintis authenticated comprises: determining a matching score that indicatesa similarity between the registered image and the input image in theoverlap region by assigning the first predetermined weight to the firstsub-region and the second predetermined weight to the second sub-region;determining an area of the overlap region, and determining whether thefingerprint is authenticated based on at least one from among thematching score and the area of the overlap region.
 18. The method ofclaim 17, wherein the matching score is determined by assigning each ofthe first predetermined weight and the second predetermined weight whena normalized cross-correlation between the registered image and theinput image is computed.
 19. The method of claim 15, wherein theregistered image comprises information that relates to minutiae includedin an original fingerprint image that has previously been input.
 20. Themethod of claim 15, further comprising: dividing, by the processor, theinput image into a plurality of first regions and dividing theregistered image into a plurality of second regions; and matching, bythe processor, the input image with the registered image to form theoverlap region, by comparing the plurality of first regions with theplurality of second regions. 21-28. (canceled)
 29. A method ofauthenticating a user based on fingerprint recognition, comprising:receiving, by a fingerprint sensor, a fingerprint of the user;receiving, by a processor, an input image that corresponds to at least aportion of the fingerprint of the user; determining, by the processor,from among a plurality of templates, whether an authentication templateto be matched with the input image is present by comparing eachrespective one from among the plurality of templates with the inputimage; and when the authentication template is determined as beingpresent, facilitating, by the processor, an authentication of thefingerprint, and updating, by the processor, the authentication templateby using the input image.
 30. The method of claim 29, wherein when theauthentication template is determined as being present, the fingerprintis authenticated, and a number of authentication successes of theauthentication template is updated.
 31. (canceled)
 32. The method ofclaim 29, further comprising: after the authentication template isupdated by using the input image, determining, by the processor, whetherat least one additional template, from among the plurality of templates,matches with the authentication template that has been updated, and whenthe at least one additional template is determined as being present,merging, by the processor, the authentication template that has beenupdated with the at least one additional template.
 33. The method ofclaim 32, wherein a first number of authentication successes of a newtemplate generated by merging the authentication template that has beenupdated with the at least one additional template is set to be equal toa sum of a second number of authentication successes of theauthentication template that has been updated and a third number ofauthentication successes of the at least one additional template. 34.(canceled)
 35. The method of claim 33, wherein the updating comprisesupdating the authentication template based on at least one from among anarea of an overlap region formed by overlapping a registered imageincluded in the authentication template with the input image, and anormalized cross-correlation of the registered image and the input imagecalculated in the overlap region.
 36. The method of claim 35, whereinthe authentication template comprises a plurality of registered imagesand the overlap region corresponds to a region in which all of theplurality of registered images and the input image overlap each other.37-41. (canceled)